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ABSTRACT 


An alternative to the normal model in time series 
analysis is presented wherein the sequence of random 
variables have exponential and mixed-exponential marginal 
distributions. The moving-average and autoregressive 
models are analysed with respect to serial correlations 
and conditional expectations. The mixed-exponential 
autoregressive model and the mixed autoregressive-moving- 
average model with exponential marginals are presented. 
A method of estimating the serial correlation coefficient 
is examined, and digital computer simulations of several 


of the models are given. 
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I. INTRODUCTION 


Gaver and Lewis (1975) have described a first-order 
autoregressive stationary sequence of random variables 
having exponentially distributed marginal distributions, 
and have given extensions to Gamma distributed processes. 
Lawrance and Lewis (1975) have described similar models 
for the moving-average process. Motivation for these models 
includes finding an alternative to the normal theory of 
time series analysis, and developing a model with correlated 
random variables with non-normal marginals. 

Several extensions of these models are also of interest, 
including a mixed-exponential autoregressive process, a 
second-order moving-average process, the kth-order moving- 
average process, and the mixed autoregressive moving-average 
process and the estimation of the parameters concerned. 

This thesis is concerned with detailing the properties of 
some of these extensions. 

Digital computer simulation of these models is also of 


interest, particularly in the estimation of parameters. 


et 


E 


II. THE EXPONENTIAL AUTOREGRESSIVE MODEL (EAR1) 


A basic stochastic model often useful in representing 
time series is the first-order autoregressive model. Unlike 
the moving-average process, discussed later, the current 
value of the first-order autoregressive process is a linear 
combination of the previous value of the process and an 
independent error term. Although similar in appearance to 
the moving-average model in that it, too, is a linear com- 
bination of random variables, a principle difference is that 
each value of the EARl sequence, regardless of order, is 
correlated (depending on the correlation coefficient) to 
every previous value. It can be written as an infinite 
moving-average over the sequence of l.l.d. error terms. 

The general form of the kth-order autoregressive model 


is 


Xi 7 ο όν "Piet αν ως), 


Only the first-order process will be considered here 
as it has great potential for describing actual series, and 
higher-order processes, with the conditions that will be 
imposed, are difficult to analyse. The first-order process, 


then, is 


vi 


^» 


= px, κε, ILL. 


If the ει are normally distributed random variables, the 
x; are a normal sequence, and this is the case which is 
explicitly or implicitly considered in the literature 
(see e.g., Anderson, 1971; Box & Jenkins, 1970). In this 
thesis the condition will be imposed that the Gu) will be 
a stationary sequence having a marginal distribution that 
is exponential with parameter λ. A sufficient condition for 
such an AR sequence to be stationary is that |p| « 1 
(Box & Jenkins, 1970). 

The determination of the error term Ej is relatively 
simple (Gaver & Lewis, 1975). Beginning with the basic 


ARl equation, and assuming that the X, sequence is stationary, 
KS Psg t Ege 
and taking the Laplace transforms 


-sx -Spx, 1 “SE 
DE T i S 


Efe fe 


sp se, 


FEE: E 
“ele ete t} 


by the independence of (es, κι 1}. Therefore, 





“sx. 
“SE, i 
" Φ. (s) - Ele ij-E.—) ., 
> v Si SPREJ) 
ele Të 
Nov the distribution of ixl is defined to be exponential 
(A), i.e., 
-sx 
Ee 3) - --ᾱ--. 
Ats 
Therefore 
$, (s) λ 
- X; - τς 
ας (E) "pn € 7 
S i Ki X + ps 
: _pstr . λ 
ert oe + Ore) pris 

It is simple to see that this is in fact the Laplace trans- 

form of a positive random variable, by inverting with 

respect tos; for 0<p<1l 

ει = 0, probability p 11.2. 

9 eir probability (1-p), 
t 


where ei is an exponential random variable with parameter λ. 


Therefore 


- 


Xi = Pej” probability p Il.3 


px, 1 + ei probability (1-0). 


It is simple to see that it is not possible to allow p to 
be hegative; since Xia is by definition positive and with 
px. negative it is heuristically clear that it is impossible 
to find an independent £i which makes κι positive. 

Owing to the nature of the model it will be seen that, 
unlike a mov-ng average process, there is a partial corre- 
lation of all orders. | In examining the joint distribution 
of x; and Χὶ 1! it is easier to analyse the joint Laplace 


transform. Thus 


ba 


+ + Ej 


PË) * & 


and 
Xp  ΡΧ ΤΕΙ Ων 
Therefore 
781X; 7 SoKi 1 -s (ρῦκι 2 + pej + £1)-85(px; jte, 1) 
Efe } = Ete = = ἘΠΕ en 
-øs μα ο πω 7816; 
= Ele jete jete. πα, 
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e^ 


by the independence of Xi 2) j- Εἰ" 


By definition 








-SX 
x λ 
Efe P rur 
and by derivation 
-se 
Efe i) = LIB, 
so that 
πα χι 783X; Atp(ps,ts,) Atps 
E(e 1.1 4 dela λ 12 1 
Atp 81*085 Atpsj ts, Msi 
MAtosj) 


» (84) (Atesj ts) 


One aspect of this equation is that if ve set $178575 
we get the Laplace transform of x, * x, ,, and it is not 
Gamma (2) because of the dependence between Xi and Χι η" 
It is Gamma (2) if p = 0, however. If either S} or 52 is 
set to zero we get the marginal exponential distributions 
of x, and κι 1) but the transform is not symmetrical in 51 
and 52: 
The first and second moments of x, and X,., can be 

obtained by taking the first and second partial derivatives 


with respect to each variable, and they are found to be the 
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v 


same as expected for a marginal exponential distribution, 


ie., Ex)  EGx, 1) = 1; EG,7) - E(x, 47} = 2/λ2. 


The variance is then yx. 
All that remains to finding the correlation is to find 


the expected value of the joint distribution. We have 


2 


39 
E(x,X, 1) = 
ics 51252 s, 704; 5,-0*. 


=a 


ex. 


λ 


Thus, since 


Cov (xi xi 1) 


Var(x,)Var(x 


u 
o 


Corr Gn x; 1) 3 
i-l 


Note that the first-order serial correlation is bounded by 
0 and 1 so that the EARl model in some respects is a more 
versatile model than the EMAl model discussed later. For 
the EMAl model the correlation is positive and bounded 
above by 1/4. 

The other serial correlations may be found in a similar 


way. All that need be computed is the joint expected value 


12 


“6 


of the lagged values of {x}. For example, p, may be deter- 
mined by finding E(x,x, 2) and replacing it in the above 
formula, although it is known from the theory of Markov 
processes that py, ow pë. It can be shown without much 


difficulty that 


k 
-8,X, - SK, λ(λ + ρ 5.) 
Ble 2 2 2"i-k, . 1 , 1.4 
(A + ϱ 89 + 52) (A + 51) 
so that 
k 
E(x,x, ,} = 5-5 1 


ik 12 i 
and hence that 


Corr (x. x, 


gua) 5ο, (Ko Or dy 2, ...) 


as surmised above. 

Nothing has been said about about conditions for 
stationarity in the EARl sequence. It is simple to check, 
however, that if the sequence starts with Xo = ο as an 
exponentially distributed random variable, and then con- 


tinues as defined by II.1 for i= 1, 2, ... with the 


error sequence II.2, then the x; sequence is stationary. 


A. CONDITIONAL EXPECTATIONS OF THE EARL SEQUENCE 
The conditional means and variances of pairs of random 


variables in the sequence Gul may be examined for comparison 


“13 


with those of other bivariate exonential distributions and 
as a potential method for evaluating the sample coefficient 
of correlation. 

In any process, the conditional expectation (mean) of 
the Xia term, given the ith is 


E{x, =th= f 


aly Kek, μι |x, Silt dx; 
5 
E Xi 47 Ff (tr Xj) AK) 





f£, (t) 
E 
The joint distribution of x, and x,,, is not available in 
a simple form. The joint Laplace transform of this joint 
distribution has, however, already been established (II.4): 
ο -s 


ez Ad = Ri e 


X. 


- 85x 
LZ 2 itle 


(x, ,X ) dx, dx 
ΧΑ Χ “il i 


itl 


Therefore, if the first partial derrivative with respect 


to the second variable of the joint Laplace is taken, 


LJ -84,X,-8,X. 
26 . g 1*1792*141 
em SS Χ 5 f 


εχεις) dx dx. 
2 0 ΧιΧ di 11 A XX 


and then 52 set to 0+, we have 
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Y» 


m 


SEXE 


ao £ 
Xixi 


E 


(x, »X. ) dx,dx, 
oar 3 SEL E ue të që 
2 82-01 


= Ji-x,,,e 
0 itl 


and it will be found to be the same form as the negative 
of the Laplace transform of the numerator of the conditional 
expectation, E(x, 441%; = t}, identified on the previous 


page, viz., 


numerator = d^ XiQ1f 0 (X244). ἄΧι γι 


ΒΚ. 


- 
Laplace transform - [fie fox. G6 X441) dx,dx; |j. 


0 iitl 


Therefore, applying this procedure to the Laplace of the 
joint distribution $, to find Ex) x = th, first 
differentiate with respect to Sor then set 52 to zero, 
invert with respect to 51, multiply by minus one, and 
finally divide by the marginal distribution of Xi for the 
desired result. 
Applying this to the EARl model yields the following, 
= — | - a λΕ(1-ϱ)/ρ 
EGG 4 xi t) Xt) l-e ) 
Nearly the same scheme may be used to find the 'backward' 


conditional expectation, EGG [i11 = t). The procedure here 
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» 


E 


is to take the partial with respect to the first variable, 
set s, to zero, invert with respect to Sor multiply by 
minus one, and divide by the marginal of Χμ’ This yields 


the following for the EARL, 


E(x,|x;, = t€) = X (ae + (1-p)} . 


Other conditional expectations may also be found in a 
similar manner. The conditional expectation of any function 
Vs Ry > gi given X is (Zehna, 1970) 


2 


E( (x5) Ix, = a ος) E (x5 |) dx, 


ud Y(x)f (x,x,) dx, 





f, σι) 


Again, the Laplace transform of the joint distribution is 
available 
x 


=8,X, = 85%, 
$zE(e ti 2d 


15 
0 


x 


1*4 7 52%i41 


X; X; 


an tel) iye 


The second partial derivative of with respect to 52 


is 
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p 


ο T84,X. - 84X. 
Eu 2°itl 
AE Ὁ £ (x κ. νι) dx,dx 
2 0 i+l XiXial (| itl i 


926 


(382) 





141" 


And setting 52 to zero yields 


2 œ -5.Χ. 
— 2 Σ Ha s FR fx x 
(352) s,=0+ 0 iv itl 





(Χμ) dx;dx;, 


This, of course, is idential to the Laplace transform of 
-5.Χ 


the numerator of EG ix) with respect to Xi, E(e 1 ij, 
Applying this procedure in a manner similar to that for 
finding the conditional means, to find the conditional 
second moment of Χμ given xj: i) take the second partial 
derivative of with respect to the second variable (52) 


ii) set s, = 0+; iii) invert with respect to 817 iv) divide 


2 
by the marginal of Xi. As with the conditional mean, the 
procedure for finding the conditional second moment in the 
'backward' sense, EG [xj 1) is the same with the first 
and second variables being reversed. 


For the EARl process, then, the two additional second 


moments are 


2 1 At) ο-λΕ(1-ϱ)/ρ 
(1-0) (1-ρ}ρ 


εἰκ; = t} = 217 


14114 
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m 


and 
2 ida. -1, 2 -2u. 
E(xj κι μι = t) = t"(2p*-p) + 4tÀ “(p-p*) + 2A “(l-p) . 


These conditional moments (and the variance which follow 
directly) present a contrast to those of the MAl process 
(presented later) in which all forms are exponential. Also, 
allowing p to approach zero or one produces different results, 
namely that as p approaches one (i.e., completely correlated 
events) the distribution becomes degenerate and all members 
of the sequence are identical to the first. If p goes to 
zero, it becomes an ordinary Poisson process. Both ideas 
are summarized in the table below. A partial check on the 
accuracy of the above is made by first multiplying by the 
conditioning marginal and then integrating. The result is 


found to be the remaining marginal's expected value. 


Term Lim {term} Lim {term} 
p70 pri 
Bix,,jixj = €) 1/λ ε 
Είκι [κιν = t) 1/4 t 
EG ix, = t) 2/12 +? 
E(x? |x, 4, = €) 2 e 
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” 


An important property of Markov sequences is that while 
the correlation between variates k apart is non-zero out to 
infinity, the correlation between Xi and Xian! given any 
value of xj between Xi and Kjyk” i.e. i < j < itk is zero. 
The zero partial correlation is a means of identifying a 


Markov sequence. 
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III. FIRST ORDER AUTOREGRESSIVE MIXED EXPONENTIAL PROCESS 


In this section, the basic autoregressive model will 
be used, but rather than demanding the marginal distribution 
of the 6) sequence be a simple exponential (A), a marginal 
distribution of the three-parameter mixed exponential 
Oir D T) will be imposed. It will be shown that for 
certain values of λα» λ2 and m, an ει Sequence exists 
which gives this marginal distribution. 

This model is far more versatile with respect to 
fitting observed distributions, particularly very skewed 
ones. With suitable choices of the parameters, distributions 
having any desired mean and coefficient of variation between 
l and e may be produced (Cox, 1962). 


Using the autoregressive scheme mentioned earlier, 


Xi ΞΟΧΙ 1 + ει, Ὁ £,p* 1, (EO, li ἂν is 


and taking the appropriate Laplace transforms, 


“Sx. “px, 18 - ES 
€, (s) “Ble 3)y-zgte 11 i 
E 


} 


“DX, 18 “E,S 
Efe isl e 1) 


“ox, 15 -ε.5 
Efe 7 l jefe + bs 


r Xi independent. 
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Therefore 


(ps) 


The probability density function of the three parameter 


mixed exponential is 


-ληχ -λρκ 
f(x) = Te * (177)A5e . 


The Laplace transform of this, and hence the transform of 


the imposed ma ginal distribution for the x,'s will be 


T (1-1) A, 
ΟΝ ΕΣ Tes 


ληλρ * s + (1-7) A38 


Aj + 9) 05 * s) 





Similarly, the transform of ΡΧΙ 1 will be 


5 + TA, ps + (1-1) AZ 9s 


πα DE t88 ve, i 
i-1 Aj t ps) O + ps) 


Then 
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Y 


6, (s) 


x 








E Në” E l 
ps 
0X4 
λ1λ2 * TA, pS + (1-1) à ps 
Ay ts 215 
- X * Th PS + Xs = TA PS 
— n + os, + ps) 
1 * 98 2 * 08 
Ν O4 tps) (A, + ps) (135 + πλι5 * λ28 πλ 55) 
= TA, + s)(A, + S) (A,A, + TA,pS + À4pS Mbps, 
1 ts 2 * 8 1^2 Ἐπ 105 + 205 EJ 205) 
" O4 + ps) (A, + es) (425 + s + A, “πλ ϱ)) 
1 + 8) OZ + 5) 0,35 + βίπλη ti - TKD 
Mz 
(A, + ps) (A + ps) (s + πετ πε) 
£ 1 2 TA, + AQ - πὰ; 
AA 
Aj * s) (A, + s)(ps + 1-2 ) 
Ἔλα Wai T AS, 
Let 
ex M^ Š Arig 
TA, t AQ 7 TAD πλι + “mA, 


Consider C as a function of T. At ns 1, C= roe and at 
nz0, C= e The derivative of the denominator is 


E - Age so that the denominator increases from àa to ET 
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u 


monotonically as π goes from 0 to 1 if El > λο. Therefore 
C decreases monotonically from A, to xo as T goes from 0 


to 1, and also λ2 <c λ 


IA 


1 and C > 0. By extension 


àl secs àa or àa «ας EL depending on which of the 


parameters Mr is larger. 


Then 
(Aj + ps) (λρ tp s)(s + C) 
te {5} τπτ τε, F Ee * pS) 
E +s + s) (6 + ps 
i 1 2 


_ (Ps + Ay) (os t AQ tË 


OQ, +s), ts) & * s) 


2 2 2 
SAN, tesë Atos Ay tes À1*€34A5-pCs 


Q4 + s) (À, + TE s) 


e + (19) d 


This is not necessarily the Laplace transform of a distribu- 
tion function; it is shown that this is the Laplace transform 
of a distribution function for certain values of the three 
parameters by explicitly inverting it. Churchill (1970) 


summarizes Heavside's equations in stating 


If f(s) is the quotient p(s)/q(s) of two 
polynomials such that q(s) has the higher 
degree and contains the factor s-a, which 


is not repeated, then the term in F(t) 
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L 


corresponding to that factor can be written 


in either of these two forms: 


at 
$(a) est or -pal e , 
q' (a) 
where $(a) is the quotient of p(a) divided 
by the product of all factors of q(s) except 
s-a. This Theorem is valid when the constant 


a is any complex number. 


Therefore, taking the inverse Laplace transform, term by 
term, using Heavside's equations and writing it as a proba- 
bilistic mixture yields, if A, Ë Agr Ελρ 7 C or pë F C, 


(term 1) with probability p, & = 0 


with probability (1-0), ει has density 


e E CC CC DG 
- =), Ag -Ἕλι Ag HE ZA, FE ZA, HCA AC == 
(term 2) (be Cte Ὃς as Ax 2 μμ ἕω} 
d Q,-90,-9 
j^'g p 


-MtOCAQA4337 0144442 * 0343435 *034 4 A, *€14A47CA,À 
(term 3) «(De + 1172 112 1132 TE 1'2 κ αἱ 
c 
(ag 7 AJ - AQ) 


-ληΈ “ALA AÀ PAA APAAL AS tPA AA tCA, Ag pCi À 
(term 4) “me κας dë Më Vë Nr vë Në κα 
c 
WT i = Ag! 
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This may be rewritten as 
-0, probability p; 


“i 


Othervise the density is 





LE {ante (C = X) (C - -λρ)ρ 
p E - 
(pA, = C)(C - pë) 
-Àt 
1 (6 - λε) (pA, - AS) 
+ àe ο a 2. 
Aj 7 AQ) (C - pë) 
7M* (= AQ (pë, - AD 
+ λρθ , probability (1-ρ), 
Aj 7 λ2) (PAZ - C) 
where 
AA 
c= E yr eee 
TAY + Q- DA, 


λ11λο >0,0<7<1, 0< p «1, with points of singularity 


at pay =C, pry = C, λρ - e 


Again, we are assuming that we have a density function 
i.e., that the function is positive and its integral equal 


to one. 
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For the case PA, = C, 


ει = 0, probability p 


-λρξ λρίλρ - ϱλ)) (QA, - Ag) 


=e <M, probability (1-1). 
eO, 7 AQ) 05 7 Xp 


For the case pia = C, 


Èp = 0, probability p 


-M.t A,(pë, - λε) (pA, - AQ) 
=e lt 1.2 1 1 2 , probability (1-p). 
ρίλῃ 7 AQ) O5 - Ag) 


For the case àl = Ag the necessary conditions for Heavside's 
equation are not met, and the inverse must be taken after 


equating Ay to X. Recalling from above, 


T Oy + ps) (λα + ps) (Ay A, + TAS + 8 - πλρ8) . 
ey (Ay + 8) (Ag + S) (λῃλρ + πλιρβ tips - mos) 


Setting A = λ2 =k, 


= (A+ ps) (A+ ps) (12 + mAs + As - màs) 
(A + s)(A + s) (12 + πλρ8 + Aps - πλρϐ) 


- At ps 
Acts 
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Finally inverting with respect to s, 


= 0, probability p 


Aet, probability (1-p). 


Fi 


Which is, naturally, the same as that for demanding Gul 
be distributed exponential, parameter A, or that m = 0, 1. 
The inverse transform, a function of t, may be verified as 
being a probability density function by showing that it 

i) integrates over the domain to unity, and ii) it is 
non-negative over the domain. Establishing the first 
condition, 


- ) 
ËS SER ot se Cute 


-C(C - Ag) (C - àa 


- C) (pà, - C) 


dt 


(pay 


-λιΈ AQ(C-24 (pA, - 215) 
1 1 1 uF 2 at 
Gj 7 Ag) (C - ϱλῃ) 


5 λος λρίς - λρ) (pë, - Xi) 


+ ü-p se at 
0 (Aj 7 AQ) (Pà; - ©) 


2) 
(pA, = CO (pàg = C) (Ay = AQ (C = pry) 


se = XJ (C — X4) (ο - X) (pA, - A 
at GSPN eg a a να 


(€ - A9) (pry - Ay) 


Gj” 13) (pA, - C) 


$ 
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“i ει dt = p + (1-0) L(C-A)) (C735) (44745) £ (C724) (634 745) (ρλρ-ς) 


+ (C19) (035714) (Ε-ρλῃ}] * (X725) (C71) (pAz-C) 
= p + (1-p) (-Cpà} 457CCp35* 0313, Not GA AL tCppA AD 
t CCA PPA 44 457CÀ JA tCCPA,-CCA, HCPA, AY 
22 1 LXX 122 


-Cpp Az -CPA A +C] 157034 Ay A5 * 00343532) 


+ (Ay = AQ) (C - ϱλ) C - ©) 


O4 = AQ) - £A) (pA, - C) 
(Ag λ2) (C = ϱλ]) (pë, - ο) 


=p + (1-0) 


The non-negativity conditions are not as readily established. 


Analysis is facilitated by forming the following cases: 


Case 1 1 pd, 50 
Case 2" : PAD € 
Case A : PAD > AL 
Case B 2 - px, « LT 
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With the previously stated conditions, EL 2 >0,0<p<il, 
and n «ος DL it is clear that case lA, 2A, and 2B are 
of concern and 1B is not well-defined. 

Beginning with case 2B (ρλ2 < C and PAD < Aye it is 
clear that each of the coefficients of the exponential terms 
is non-negative and hence, over that parameter range, the 
function is non-negative. 

For case 2A, the first and second terms are non-negative 
and the third non-positive. If the exponent of the second 
term is reduced from -Àt to -Ct/p this will decrease the 
value of the positive term. Similarly, if the exponent of 
the third term is increased from τλρε to -Ct/p, the value 
of this negative term will be decreased. Thus, the resulting 
function will everywhere be of lesser or equal value to 


the original form. In terms of equations, 


εἰς = A) (C = Ay) të AC- X) (pA, - AQ) 


> εἰ = e CUP 
$ (C - pà) (039 - C) (Aj - X9) (C - pë) 


λρίς = λ2) (PAZ - Ag) 
(y= AQ) (PA, - ©) 


* 
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πι 


= e£. (ο (ο-λῃ) (C4) 04715) ἐλ (6-λ1) (024745) (9A57C) 
+ λο(0-λ2) (93573) (C= ϱλ)) . 


E O4 - 13) (pë, -c(c- 0^) 


ect/e (5 7 X) (C = pà) (C - 934) 05 + AY - C) 


Gj 7 33) (ρλρ - C) (C - pë) 


se u.a -020 


1 
The fourth case, 0^, > C, cannot explicitly be demonstrated 

to be everywhere non-negative, although it is so at t = 0 

and as t approaches infinity. Even without considering this 

case, the model has great versatility, particularly with 


small values of p. 
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IV. ESTIMATION OF p FOR THE EXPONENTIAL 
FIRST-ORDER AUTOREGRESSIVE MODEL 


Using the exponential ARI model defined above, 


Exp (1) jeg; 305 x 
" Γ 
τ. 
m ἢ 
- së ij , 


where εἰ is 0 with probability p, and exponential \ with 


probability (1-0). Now if a new random variable is defined 





as 
2, dw 33:2 ων 
then, 
2 Xie “E "Yn εἰ e 
Ki *i-1 


Again, if the x,'s are distributed Exp (1), 





j probability (1-9) 


p ' probability p 
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Clearly, if the sample is large enough and/or p is large 
enough, some of the z's will "estimate" p "exactly." 

Now in a sequential estimation situation one only has 
to observe the minimum of successive 2;'s, and as soon as 
a match is observed between a current value of z, and the 
minimum of the previous z's, that value is exactly p. The 
time to the first occurrence of the value p in the sequence 
Zi is geometric with parameter p and the time to the second 
match, say M, is the sum of two independent geometric 
random variables. This is a negative binomial random 
variable and E(M) = 2/p. 

It is also clear that once p is known it is possible 
to unravel the εἰ error sequence, and there will be M-2 
(a random number) of exponential variates from which to 
estimate the exponential parameter 4. This can be done in 
the usual way, and more observations can be taken to achieve 
any desired precision in the estimate of i. 

For fixed times of observation, say from Xo to Xii 
giving n observed values of Zi, there are three possibilities: 

(1) A match occurs in determining the minimum Zi 
sequence, so that p is known exactly, as above. The probability 
of this is 1 - (1-90)? - npl(1-p)"^l, which will be very 
close to one for p ~ 0.5 and n large. 


(2) Either a) for exactly one i, x, 


q 7px 


i-1 
(probability np(1-p)?71) or 
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b) for no i do we have X; = PX 1 
(probability (1-p)"). 
In the second case it seems fairly clear that a reason- 
able estimator of p, with a positive bias, will be min (21). 
Note that the ratio Εἰ/Χι 1 has an F-distribution, xim 


F5,2' where 


n/2-1 


+ 
y 


vs [8 


2 
2 (l1 + ny/m) (+m) /2 

Since the ratio is independent of λ, so will any estimator 
of p based on the 215. 

It is possible to get an exact estimator of p, with 
known bias term, by averaging the z,'s. The variance is of 
order l/n, whereas it is conjectured that the variance of 
the estimator based on the minimum of the 218 is of order 
1/22. 

Some simulation studies of the (min z,) estimator are 
given in the Appendix. The simulations tend to substantiate 
the speculation that the distribution of the minimum (21) 
(when it is not an "exact" match of p) is exponential 
(p,n), the two parameter exponential distribution with 
mean = p + 1/n and minimum value p. (Johnson & Kotz, 1970). 


It would thus appear that 
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Min (z) = p , probability 1 - (1-9)? - np(i-p)^! 
n-1 


Exp (p,n) , probability (1-0)? + np(1-p) 
The expected value of this is 
E(min(z,)) = E(p) = (1-(1-9)"-np (1-9)? p ( (1-9) 


+ np(1-p)71) (p+1/n) 


with a lower bound of p. 
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V. FIRST-ORDER EXPONENTIAL MOVING AVERAGE PROCESS 


The general moving average model of time series analysis 
(see e.g. Box & Jenkins, 1970) may be described as a 
sequence of random variables formed as the weighted average 
of a number of previous observations. It should be noted 
that it is not a true "average" in that the weighting 
coefficients do not sum to unity (or any other specified 


number). 


XQ = Bye; + βρει] tt + BkEi+k-1 * Fitk (i=0,+1,42,...), 


(-l < Bi « 1) 


This is one of the two simplest forms of times series 
models (the other being the autoregressive model, discussed 
previously), and hence is relatively straightforward in 
terms of calculations and correspondingly limited in terms 
of describing natural phenomena. As the number of terms 
in the sequence is finite, the sequence may be shown to 
be stationary (Box & Jenkins, 1970). 

Considering, at first, the first-order forward moving 


average process, 


x. = Be, + Ejil (i = 0, 41. 32. soj -1« B « 1) 
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it is clear that there is first-order dependence, but 
nothing further (i.e., x, may be correlated to Xii! depending 


on the correlation parameter, but not to x This first 


i-2)* 
order dependence is different from the Markovian ARl model, 
where x, and x; 2 are conditionally independent. The model 
is described as "forward" to distinguish it from the 


"backward" process, 


xy = Bey Gj. qi 


whose properties are similar. Note that if the i.i.d. 
sequence {ει} is normally distributed, x, will be normally 
distributed as well. 

If the condition is imposed that the marginal distri- 
bution of the x's be exponential parameter 4, the EARL 
model shows that we may construct a first-order moving 


average as 


" probability β 
Bej + ειμι, probability (1-8) 


Verifying that the distribution of xi) is Exponential (A), 


the Laplace transform of the distribution is evaluated 


as 
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L(£, (x,)) = £* (s) 
Xi i Xi 


-5X 


ΞΕίο +} 


~sBe 


1 “she, - 
“ele Dere i 


se 
i+1) (1-8) 


-8βε -ββε, “se 
“ele Άβα βίο ele ig, 


by the independence of the terms of (c,). Thus, for any 


i, 


BA , (1-82 


f* (s) 
τ A+s (A+ sB)(s + A) 





Ats 


rat 


Lie Q.E.D. 


The first serial correlation 61 is defined in the 


conventional manner: 


EJ 


* E{x,x,,,} - E{x,}E{x,,,} 

ob πριμ ER 
var{x, JVar(x, 9) 

The expected value of the joint distribution of (κιν) 

may be found in a number of ways. Two of these are i) to 


examine the form produced by the product and ii) to find 
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the first mixed partial derivative of the double Laplace 


> transform of the joint distribution evaluated at s = 0+. 


Both techniques will be used here as the results will be 


useful later. 


We have 


κ; = Bes, 


. tet 
Be; Ej 1 


κια θεα e 
- Beis * Sine 
The product, then, is 


2 
ἘΚ T B SjSjqa 


Bej(Be,,) + 


1 


P 


(Be; + £143) (Be) * ειν), probability (1-8) 
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probability β 


probability (1-8) 


probability β 


probability (1-8) 


probability 82 


ει κο) ' probability 8(1-8) 


(Be; + £141) Bes μι’ probability (1-8)8 


2 


=) 


2 
E(x,xy,1) = BE(B7eze443} 


+ 


B (1-B)E{B ese μι + 86,6142) 


+ 


2 2 
(1-6) BE{B 6,6541 + 861,3) 


+ 


2,2 2 
(1-8) E{B ejejça + Begeja + Beis] + 63416442) 


Defining y = Ε{ει}, and g? z Var(ei), then, by stationarity, 
u = E(e,,,) and e? = Var(c,,,). Now, 

2 à 
Etejej,j) - Efe, EC 04,4) =u, j #0, by the independence 
of the terms of {ει}, whereas Elej,jejjj) - i? + ο, by 


definition. With this in mind, 


B(x,%5 41) = 8 o^ 


Β(1-β) (Bu? + Bu?) 


+ 


+ (1-8) 8(87u? + sa? + c?) 
+ (1-8) “(822 «a * 8G? to) + uê) 


a-go? + μὲ. 


Therefore, 


M e τ BOG IE Ou) 


ρ 
1 var{x, }Var{x;,,} 
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së β(1-β)σ2 + μὲ - u2 


a?o? 


B (1-8) 


It should be noted that {ει} is distributed exponential 
parameter λ, so that σε, - Me, = 174, and this could have 
been substituted earlier. This substitution will be useful 
in considering higher-order moving-average processes. The 
correlation P) is the most limiting aspect of the model as 
the correlation of the x's is strictly non-negative and 
bounded above by 1/4, as compared with the conventional 
(i.e., normal) MAl model where -1/2 « 01 5 1/2. A second 
problem associated with this EMAl model is the symmetry 
of the functional form about 8 = 1/2. For purposes of 
estimation, differentiation between a coefficient β of, 
say, .15 and .85 will be impossible if estimat on is based 
on pi. As will be shown later, higher-order EMA models are 
similarly limited. By the nature of the model, all higher- 
order correlations in the EMAl model are zero. 
A.  LAPLACE TRANSFORM OP THE JOINT INTEKVALS IN THE 
EXPONENTIAL MOVING AVERAGE MODEL 


The basic model is still 


x, = Be, , probability B (0 « B « 1, 


Be. te probability (1-8) δη 83 ee RAM 


i 342^ 
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and the product random variable is 


probability 
= e? 
ΧΙΙ 86,8644, 
Bei(8ei,, + Eiga? β(1-β) 
(Be; + €5,3)B8e54) (1-8) 8 
8 (Be, + €,,,) (86,4. + Ej ja) (1-8)? 
i itl itl it2 - 


Taking the double Laplace transform, we have 


784.X, 7 S,X, 
Id 2 11 
$ (51,52) = Efe } 
Xixi 1'2 
sage Ἔα,” Sa 
=s} ße, - s, (ße, E Ba) 
+ Α(1-β)Εί6 ἡ κ μα ἢ 2 itl it2) 
=s, (Be, te,,.) - s Be 
+ (1-8) 8Ele 1 i itl 2 in 
oue απ Ῥ qq tË ua + 642) 
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7548€, - SoBe 
(e 152) - B”Ete 1"*i 2 itl, 
iua 1 


7848€; - Soft; SE 
+ g(1-g)E(e gri Pë 2711] 


-SiBe; - (51  82β)ε. 
+ “-ρ)ρεία T i * 9285014, 


-848c6, = (8, + 858)6,,4 - 8,€,,,5 
+ (1-8) 2Ete Di 1 205/8141 2334421 


7818€ “SoBe, 
= geje l'iyge 2 id 


-5-βε:- -5.,βε: -S5€ 
*gü-pE(e | πο 2 ig 2112] 


cte) + 8280447 


+ (1-g)gE(e 


-(s,ts.B)e, 
jete 172 itl 


58e οφ 


+ (1-8) 2Ε(9 igte jete 


by the independence of {ει}: With ey > exponential (2), 


-. HË λ λ 
Pr, tto? = SETTA T μετ] pe, T $57! 


λ λ λ 
* g(i-g) res, res ey! 


À λ 
* (1-8) res, re, res 
2 λ λ λ 
+ 4 Gje reres 5x x 
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ο ee ο 3,3 2.3 2.2 
re ae = (BA + BA's, + BAS) + B A s3 + BA 5152 


+ BA a? + ant + 83s, + ps, E β2λ4 E Be, » 8333s 


art + 62035, + ns, + βᾶλρο,2 - gh! - 7275, - ghi? 


+ 


2 4 


$ Xt + 8313s, us pis, - βλ. 


3,2 
BA 52 


2,3 
= BN $5 4 g2jë 4 8532s.) 


E 


((A + 85) 0 + 885)0 ts, + s58)) 


4 


3 2 
+ gas) * 8717515, + gas) + βλ 52 +B 178 2 


2 
(X + Bs) (A + 855) (A + 89) (A ts, + 858) 


+ Ba so 





2 
A (A + Bs, + 855) 


(At 8s) C * 52) 0 * 5 * 528) 


This result may be partially verified by setting either of 
the s's to zero; this yields the Laplace transform of a 

single exponential (A) distribution. Further, if β is set 
to zero or one, the result is the Laplace transform of the 


sum of two independent exponentials or an Erlang (2,4). 
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B. CONDITIONAL EXPECTATIONS OF THE ΕΜΑΙ SEQUENCE 
The.conditional means and variances are found in the 


same manner as the EARl model, and only the results are 


provided: 
_ x ek 1-28 8. ~At(1-B)/B 
E(x, 411%; = t} =A (8t ΞΕ TY e ) 


x: ea” 2 atAt(1-8)78 
E(x; IK) -t)-221 (010*8-e ) 


2.23 
2x7 ds? 3 (1-28) XË 4 1- 38128 +6 


(1-8) 


M 


2 2 
Είχε γι Ix, = t) 


2 
+ B-8-8^) Q-(1-8)Ae/8 | 
(1-8) 


-2,148-8? _ 148-8? ο- (1-8) At/B 


2 
Eix, “|x. = t} = 2λ 
X itl 1-8 1-8 


-2E ο- (1-8) Mt/8 ] 


The conditional variances follow naturally, but none 
of these expectations provides much assistance in the 
estimation problem as each of the above expressions is 
relatively insensitive to changes in β. 

C. SERIAL DEPENDENCE AND CONDITIONAL CORRELATION IN THE 

EMAl MODEL 

The conditional correlation shows some very interesting 


properties of the EMAl process. In the EARI it was shown 
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that there is some dependence in every term of the sequence 
with every other term. In the EMAl, the dependence is far 
more limited. The κ. 1 term is, depending on the value of 
the correlation coefficient, correlated with the x, and Xi.2 
terms, but on no others, and the Χρ term is correlated 
with the x: and Xi12 terms, but no others. However, the 
κι term is correlated with both the κ and Xj] terms. 
Thus, in the EMAl process, there is no 'transitivity law' 
with respect to serial correlations in that although each 
Xii and Χμ depend on Xir αι Ὶ and Χμ are independent. 
in any event, unlike the ARI process, the EMAl is not 
Markovian as E(x, ΧΙΙ 1} is not the same as E[x, |, |xi). 
The reason for this becomes obvious when considering the 
basic construction of the model. 

It appears, then, that a conditional correlation 
involving Xi. Ἀπ and Χμ might be of interest in 
examining what this dependence is. Choosing one of the 
three possibilities, then, of Corr(x; j,x,,1|x, - t) 
as a measure, we define this as ρ{6), 

EOS XX, = t} - EG |X; = CEOs IX, = €) 


Po (t) = 
{var (Xj; |X; = t) Var(X,,i|]X, = t)) 


The covariance Covix; ΙΧ µη |Χι. = t), the numerator of the 
above expression, is sufficient for a cursory examination. 


All but E(X; Xi ντ] ši = t) has been previously calcul ted. 


45 


The triple Laplace transform of the joint p.d.f. of 
Hy i Χρ is found in a similar manner to the double 
transform found earlier, 


“ SK: 7 S4X, 


TS4X. 1 
fs** (5115215334) = Ele “I ed ΣΙ 


ΧΑΚ ida ΕΙ 


- 
αρ ει "η 
= e x. GG Xi kj) dX 


(oi Re 30 
0 ΧΑ ΣΧ) i-1'" i 111 


The mixed partial derivatives with respect so 51 and 52 

are taken of this, and 51 and 5η are set to zero. This form 
is then inverted with respect to S and divided by the 
marginal of Xi. After subtraction of the product of the 
conditional means, the conditional covariance remains, 


82 i -(1-β) λε/β 
Cov(X; iX; 441%; =t} = πε’ {(1-6) At-Ble 


-2(1-β) λε/β 
"τε 


Although non-zero, the values of the conditional correlation 
are small and are of no apparent aid in the estimation of 


the correlation coefficient. 


46 


" 


VI. THE KTH-ORDER EXPONENTIAL MOVING AVERAGE MODEL 


The basis for the first-order moving average model of 
the previous section was the solution for the form of the 
distribution of components of the double infinite error 
sequence in the autoregressive model. In that model, however, 
as applied to the exponential case, the unweighted term in 
the moving average is just exponential. This suggests 
making it a moving average of two further e's, i.e., fiii 
and £142* Using this iterative procedure we get a moving 
average of any desired order. 


In observing the form of the first few exponential 


models, a very clear pattern of progression is noted: 


EMAl x, = Be, probability β 

Bej * Ε probability (1-8) 
EMA2 Xi = Bjej probability 8) 

Bye; + βρει yy probability (1-βῃ)β2 


probability (1-8,) (1-82 


Bye; + Besa) * 5142 By 
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EMA3 x; = 816: probability By 


816, + βρει yy probability (1781) 85 

816; + 858141 * 838142 probability (1784) (1785) 84 
β1ει + βρει] + 84€,,5 Probability (1-84) (1-85) (1-8, 
it3 


And so forth. These may all be verified as forming probability 
distributions. Moreover, the procedure is not specific to 
exponential moving averages: all that is necessary is that 
for given marginal distribution of the x's, the €; sequence 
be that form which provides a solution for the first order 
autoregressive model. 

Now, rather than looking at these as probabilities and 
associated terms, they may be analysed as terms with 
associated probabilities, that is, the Bye; term appears in 
all models with probability 1. Similarly the 828141 term 
appears in all models (k > 2) with probability (1 - 81); 
etc. Complete generalization fails only in describing the 
final term. The condition must be provided that a coeffi- 
cient is not present in the final (i.e., k*lst) term of 


the model. 
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The use of a series of random indicator functions, p i 


permits the complete description of the model with complete 


generality to all orders in a closed form. Thus, for 


nal, ..., ky 


(n) 


u 


0, probability Ba 


1, probability (1-8), 


where i refers to the ith term of the nth independent 
sequence of independent Bernoulli random variables. So, 


1, M ~ i.i.d. Bernoulli (1, probability 1-8,, 0 otherwise). 


(0) 


i 


Define I: to be identically 1 and 8.1 to be identically 


1 for all i. 
Using this notation, the coefficient of the Ej term 


(for the Xi element) is simply By The second term is 


(1) (1) 
BI; fia]! the third 841i 


then, the kth-order moving average process is given by 


£142! etc. In closed form, 


k j 
(n) 
w E Ῥβιηξια πχ, ͵ 
150 411.114 π-ο * 
where i is the serial number of the ith element of the 
series, k is the order of the process, and j and n are 
indices. For example, expanding this for the x,th element 


for the EMA3, we get 
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(1) 


z (1), (2) 
X, = Bjej + B51; 6,44 * B3; 1. €, 


412 


(1) 


(2), (3) 
+I, ig Dy 


I Fit2 
The general form above can be established through mathemati- 
cal induction with the expression for k - 3 and k - k above; 


the expression for k = k+l is 


k j ktl 
(n) (n) 
κ.ε Ὁ Βέρα μα +B € 1 X. 
i^ jo J*l it pog i k+2fi+k+l ορ Í 
ktl j (n) 
- ^ ta T - 
950 131393 Aii 


The purpose in the creation of this model is to provide 
models for data with longer dependence than that obtained 
with the first-order model and to examine any tendencies of 
the upper bound on the serial correlations to increase. 

As mentioned earlier, using the standard formula for serial 


correlation, 


ἀρ E(X,X; 1) - E(x, JE{X,,.} 


J 
Var{x;} varíx,,,) 
the only non-zero contributions to this correlation will be 
a term in the joint expected value that is not present in 
the product of the expected value of the marginals. In 


terms of the model, this will occur only where there is a 
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product of two identical error terms, e.g., 


ele }Ξ ECe,JECe Lu) = vw, j #0, by the independence 


ifi+j E 
of {ει}, whereas ECC 456,4) - μὲ * σ΄, be definition, and 


in the case where {ει} - Exponential (2), 


- 2 = Li 
ECC 4,494447 = 2/n 2 ECe, JE Ce γα). Therefore, if the 
term C6;,58143 were to appear, its contribution to the 
covariance would be ca? + c?) - cu? = co”, and the 


contribution to the correlation vould be el? = C. The 
contribution of σειει aye however, would be zero. 
Thus, for example, P1 for the MA2 process is found as 


follows: 


a) 2 


(1) 
si ti it2 


KG Byey 9 BAT. “eja, E Τα. 


(1) 


E o) 
kip 7 eisi + P9144 8342 


itl 


(2) 


KE 141 ε 


I 113 


This gives 
P 7 8185 (1-8) * 85 (1781) (1-87) (13785) . 
From this example, it is easy to note the pattern for 
the serial correlations. This pattern may be condensed 


for all serial correlations of all orders of the exponential 


moving average process as 
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k-jtl  i-1 itj-l 
= - ij - <j< 
Pj,k iis Bi i 85) 143 BN (1 E l1sj$sk 
where k is the order, j is the degree of serial correlation 
and i, n, m indices. For a moving-average model, Pj KË 0 
, 
for j > k. 


Examining a few special cases, 


1 4-1 itk-1 
py, = POR Π (1-8) 84 T (1-8) 
kk gainso ἢ itk peo n 
k 
= BB I (1-8 ) 
1 kti πρ m 
= By By y (1-8) (1-85)... (1-8) 
= Bi (1784) (1785) ... G-8,) ' 
as Berl is defined to be 1. 


With 0 « 8 « 1, this correlation is clearly limited to 
ος Pkk £ 1/4. This may be obtained by setting By to 1/2 and 
all others to zero. For notational purposes, this maximum 
is achieved with the beta k-tuple of (.5,0,0,...,0). Any 
change in the values of the other beta values will cause a 
decrease in the value of Pkk* 

The first serial correlation coefficients of the first 


two orders are given by 
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ì-1+1 4-3, itl-1 
ρα”. QEQ δὲ Mea Baas ES ën 


= 8, (1 - Bp. 


as previously noted, and 


2 i-1 i 
0,27 EB, T (1-5,)8 n (1-8 
1,2 i=] Í neo n itl meo π 
= 8,85 1-8,) (< 1/4) 


+ B, (1-8,) (2-8,) (1-85). (< 1/4) 


B, (1-8) H. - B, (1-8))) 


Letting A - 85 (1-81), then Bios = A(1-A), with a maximum 
' 
of 174 at A= 1/2 = 850-8). 
The second serial correlation coefficients of the 
second and third order processes are given by 
= B3 (178) (1-85) , 


92,2 ^ Pk,k o 


which has a maximum value of 1/4 at B - (.5,0), and 
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2 i-1 itl 
Σ B E (1-8,)8 E (1-8) 
is1 i n-0 n2 πρ m 


02,3 


8,85 (1-8) (1-83) 


+ 85 (1-84) (1-βῃ) (1-85) (1-85) 


8318) (1-8,) (1-82) } 
+ (1-63) {β2 (1-81) (1-8) 1-85). , 


which may be regarded as a convex combination (or weighted 
average) of the two terms in braces, the mixture being 
determined by the value of 84- The term 83 is not present 
in the terms in braces. The maximum value of each term in 
braces is 174 (although not attained at the same time). 
The maximum value for P2,3 is thus no greater than a veighted 
average of 1/4 and 174, or 1/4. This technique will be used 
to establish the maximum value of p for a number of classes 
of serial correlations. 

One such class of correlations is that of the (k-1)st 


serial correlation, 


k-(k-1)41 i-l itk-1-1 
Preaek o ΙΔ Bi pro Bn) Bitk-1 mig ον δα) Em 
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2 i-1 itk-2 
Pk- = £8; M (1-80) Bi+k- E (1-8) 
k-1,k i=1 3 no n itk-l πρ m 


= 8,8, (1-83) (1-85) .. - (1-8, 1) 


+ By (178,) (1-8) (1-85) «++ (1-8, 1) (1-8) 


" 


By {By (1-41) (1785)... (1-8, 2) 


+ Qe) {βρ{1-βῃ) (1-81) (1785) ... (ej i) }, 
(k > 2) 
which is also a convex combination of terms each of whose 


maximum is 1/4. Another class of correlations is that of 


the (k-2)nd serial correlation, 


k-(k-2)41 1-1 i*(k-2)-1 
Pk-2,k "o që Bi an Breezy) (123) 
3 i i+k-3 
θα Beebe m. CER 3/4 
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Pena, = P181 (1785) (1-89). i78 2) 
+ B, (1-84) BË (1-8,) (1-82)... (1-849) (1-8, 1) 


+ 85(1-8,) (1-62) (1-8) (2-82) . . - (1-8, 2) (1-8, |) (1-8) 


= By L1 4 (1-84) (1-82)... (1-8, 2) } 
+ (1-8, 1) 8518, (1-81) (1-βῃ) (1-82)... (1-8, 2) 
+ ᾱ-βκ 3) (1-8) (8, (1-8,) (4-82) (1-8) (1-82) (1-84) 
“(SB ds 


As above, this is a convex combination of terms each of whose 
maximum is 1/4. In this case, there are three terms, but 
the "weights" still sum to unity. Therefore, this class of 
serial correlations is also bounded above by 1/4. 

A somewhat broader class of serial correlations is that 
where 2j > k+l. These correlations may also be specified 
and a fixed upper bound obtained. Using the general expression 
for correlations in the EMA model above, the individual 


terms are 
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ρα κ” 8183,44 (1781) (1-82)... (17 8,4) 


+ 


82 (1-83) Bo 45 (1-83) (1782) .. (1-84) (1-85 44) 


+ 


83(178,) (1-83) 83,4 (1-81) (1-8, ++ (1-854) (1-8542) 


+ 


Bjen (0785) + + + (1B 54) By 178) eee (1583) 00 (178 1) 


+ 


Bk-j+1 (1781) - (178, 4) (1787) + e e (1785)... (178, 1) (1-86) 


With the added restriction imposed that 2j > kt 1, the 


above may be factored into another convex combination form: 
P4,k = 8144184 0785) 0785) «+= (1-85) } 
* (1-8) 45) B24; €82 178,) 782)... (1784) 


+ 078,44) 078544) 83,4085 (1-83) (1785)... (1-85) ) 


* 785,5) 4-82) IT 


κεν 078) 
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Pjk = 784,44] 078544) 078344 oo ᾱ-βκ 4) CB) Kë 543 78) 


se: (1-8,)] 


3 
This form also has a maximum value of 1/4, being the 
combination of terms each of whose maximum is 1/4. 

Analysis of the remaining serial correlations of the 
various orders of the EMA models is more difficult. The 
following observations are made about the remaining correla- 
tions and a speculation with respect to the implications of 
these observations follows. Briefly, it appears as though 
the maximum serial correlation of any order for any of these 
models is 1/4. 

A. Each serial correlation of each exponential 
moving average model contains k-jtl terms, each being the 
product of various elements of 8: and (1-81) »o(421,2,...,X). 

B. Each of the individual terms is non-negative, 
bounded above by 1/4 and below by 0. 

C. When any one term is maximized (i.e., the B 
vector is adjusted to yield 1/4), the remaining terms are 
each 0. 

D. The sum of up to k-j terms can be shown to be 
bounded by 1/4. 

E. When terms are one-by-one maximized, the overall 
serial correlation may be shown to be a local maximum. 

With the above established, it is speculated that the 


maximum value for any of the serial correlations is 1/4. 
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This is borne out experimentally for EMA3 and EMA4. The 
scheme indicated below shows how the k-j+l maxima for each 
correlation may be obtained. The general formula for 


serial correlation in the EMA model, again, is 


k-jtl i-l itj-1 
pj y = = B, N (1-β)8 I (1-8). 
Jk o jer iq ἢ itj po π 


It should be noted in this formula that there are exactly 
two beta elements in each term and that they are not the 
same. Further, the Bi element, present in all terms, is 
matched once and only once by a (1-81) element from the 
second product. This is the sole restriction of the maximum 
value on the individual term, as all other elements are 
mentioned without their complements. 

Using the above observations, it is clear to see that 


each term is maximized in the following manner: 


B, 7.5 


bj € Big Ὁ 


Bisa > 8343-17 9 
8145 = 2 
βρει > Bk = (Arbitrary — not mentioned 


in the term) 
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Establishing this beta vector for the ith term will drive 
the remaining terms to zero. The itj-l terms except the 
ith each has a Bi element that has been set to zero. All 
terms itl through k-j+l have a (1-B3 45) element which is 
zero, owing to the first product in the general form, since 
the i in this notation is at least one greater than the i 
representing the ith term that has been maximized. 
The following table summarizes the establishment of 


the maximum serial correlations (each maximum is 1/4). 
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Serial Correlations 


2 x 
2 x x 
EMA 3 E x x 
Order 4 E E x x 
5 8 S 8 X x 
6 8 8 8 Χ x x 
7 8 8 8 x x x x 
8 8 8 8 5 x x X X 
9 8 8 8 8 x x x x x 
10 8 8 S s 8 x x x x x 
X - An exact analytic solution obtained for maximum 
correlation 
E - Demonstrated experimentally 
S - Speculated 
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APPENDIX 


SIMULATIONS 


INTRODUCTION 

Chapter 4 addresses the question of estimating the 
correlation parameter in the EARl model. Due to the 
symmetric nature of the expected correlation, the conven- 
tional method of estimating the serial correlation is less 
than adequate. The following simulations apply the procedure 
outlined in Chapter 4. 

The simulations were performed in APL/360 at the Computer 
Center at the Naval Postgraduate School. A brief description 


of the functions used follows. 


PROCEDURE 
Three simulations were made of the test procedure 
outlined in Chapter 4. The generated samples each contained 
500 EAR1 sequences with mean 1. The first two were generated 
with a correlation coefficient of .1 and the third with 
one of .05. Sequence lengths are 10, 5, and 20, respectively. 
In each case, a new set of random variables is formed 
as zi - Xi. Xi The minimum of these from each sequence 
is recorded and the remainder discarded. The probability 
that this minimum estimates the correlation coefficient 
"exactly" is given in Chapter 4, and only the distribution 


of the non-exact estimates are of real interest in the 
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simulation. The "exact" estimates are therefore censored, 
and the remaining sample examined for exponential tendencies. 

Moments of these samples are calculated and the values 
are plotted as an empirical log survivor curve (Cox & Lewis, 
1966). The plotted curves are very nearly linear which is 
indicative of an underlying exponential distribution. 
Johnson & Kotz (1970) give as maximum likelihood estimates 
for the two parameters of a two-parameter exponential, 

6 = Min (x,) and G8 -X- 8. The results are summarized 
in the table below. 

No further tests were conducted on these simulations; 
however, a fourth simulation was made by generating 500 
random exponential deviates with mean of .1. These were 
censored at the .1 point and were examined as above. The 
results are nearly identical to the first simulation, adding 
credibility to the speculation that the distribution of 


the minimums is the two parameter exponential (n,p). 
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EARl Simulation Summary 


Simulation number 


Sample size 


Number of terms in 
each EARl sequence 


Marginal exponential 
parameter 


Correlation coefficient 
Censored sample size 
Sample mean 

Censoring point 

Minimum value (- 6) 

ML Estimate of c (- X-6) 
Hypothesized value of 


o (=1/number in 
sequence) 


500 


10 


“1 
201 
“192 
«1 
“1 
.092 
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II 


500 


«1 
306 
«307 
«1 
«1 
«207 


III 


400 


20 


.05 
159 

“1 
«05 
«05 
«05 
«05 


IV 


500 


201 
.203 
“1 
«1 
«103 
(.1) 


USER-DEFINED APL FUNCTIONS 
The following functions were employed in the simulations 
conducted, and listed at the end of the output. 
A. n EXVAR 1 - Exponential random number generator. 
n is the desired sample size of parameter 1 exponentials. 
B. n BERN k - Bernoulli random number generator. n 
is the desired sample size of random variates of parameter k. 
C. n ARlll parm -  EARl random sequence generator. n 
is the desired number of random sequences with parameters 
parm. parm is a three-element vector of parameters where 
parm(1) is the parameter for the marginal exponential, 
parm(2) is the number of terms in the sequence, and parm(3) 
is the correlation coefficient used to generate the sequence. 
D. ZQUOT m - Takes a matrix consisting of a series 
of sequences and returns a matrix such that 54 = Xi41,43/7*4,3* 
E. LOGSURV n - Generates a vector of length n 
for use as an axis in plotting the log survivor function. 
Computes z = log (1-(i/n*1)) for a vector. 
The remaining functions used are standard APL/360 library 


functions and primatives. 
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